Cargo cult prompting

How much of the prompt engineering is Cargo cult?

The result of a prompting depends on many factors: the version of the model (cf. new GPT allows reproducible results), randomness (temperature), and the prompt itself, including the interactive history of the “conversations”. If a prompt changes in multiple places, especially in the chat setting, it is nontrivial to attribute the change in the results to a specific change in the prompt.

It may be interesting to let first-time users to spend some time with LLMs with the objective of figuring out the best ways to prompt. And then examine how they converge or diverse.